Skip to content

feat(ios): support explicit iOS simulator keychain reset - #2345

Open
thymikee wants to merge 2 commits into
mainfrom
claude/agent-device-issue-2282-088ee0
Open

feat(ios): support explicit iOS simulator keychain reset#2345
thymikee wants to merge 2 commits into
mainfrom
claude/agent-device-issue-2282-088ee0

Conversation

@thymikee

@thymikee thymikee commented Sep 6, 2026

Copy link
Copy Markdown
Member

Summary

  • Fixes Support explicit iOS simulator keychain reset for fresh-install testing #2282: settings clear-app-state clears the app's data container but never touched Keychain, so a customer's fresh-install reset left the app signed in via Firebase auth even though their in-app "Reset to Fresh Install" button cleared it.
  • Adds settings reset-keychain clear, a new iOS-simulator-only settings action that runs xcrun simctl keychain <device> reset.
  • Kept it a separate, explicit command rather than an option on clear-app-state: simctl has no per-app keychain reset, only a whole-simulator one, so folding it into the existing per-app command would silently widen its blast radius. reset-keychain documents and requires an explicit opt-in to that wider scope instead.
  • Works through the daemon/remote path for free — settings already dispatches generically to the platform's setSetting operation, so no daemon wire changes were needed beyond the shared contracts/usage-string updates.
  • Documented in website/docs/docs/commands.md and in the CLI's own --help text, including the "pair with clear-app-state for a full fresh-install reset" guidance the issue asked for.
  • Split the pre-existing apps.test.ts and snapshot-handler.test.ts suites along the app-settings.ts / snapshot-settings.ts modules they actually test — both were already over the repo's test-file-size tripwire and could not grow to hold the new tests.

Test plan

  • pnpm typecheck
  • pnpm lint
  • pnpm test:unit (full suite, 9061 passed)
  • Live validation: built the CLI, booted a disposable iOS simulator, ran agent-device settings reset-keychain clear --udid <udid> against it — succeeded and returned the expected { scope: 'simulator', cleared: true } payload; confirmed an invalid state (nope) is rejected before touching the device. Disposable simulator deleted afterward.

`settings clear-app-state` never touched keychain-backed credentials
(e.g. Firebase auth), so a customer's fresh-install reset via the CLI
left an app signed in when their in-app reset button did not (#2282).

simctl exposes no per-app keychain reset, only a whole-simulator one
(`simctl keychain <device> reset`), so this ships as a separate,
explicit `settings reset-keychain clear` command rather than folding
it into `clear-app-state` — callers opt in knowing the scope is the
whole simulator, not just the app under test.

Split the pre-existing `apps.test.ts` and `snapshot-handler.test.ts`
suites along the `app-settings.ts`/`snapshot-settings.ts` modules they
actually mirror, since both were already over the test-file-size
tripwire and could not grow further.
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://callstack.github.io/agent-device/pr-preview/pr-2345/

Built to branch gh-pages at 2026-09-06 09:20 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

Size Report

Metric Base Current Diff
Installed (including dependencies) 4.49 MB 4.49 MB +1.0 kB
Package (unpacked) 4.49 MB 4.49 MB +1.0 kB
Package (download) 1.33 MB 1.33 MB +449 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 30.1 ms 30.4 ms +0.3 ms
CLI --help 83.3 ms 80.8 ms -2.4 ms

@thymikee

thymikee commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

Please reject app-scoped input before resetting anything. The CLI accepts settings reset-keychain clear com.example.app and drops the extra argument; the direct daemon parser also accepts it. That can turn an apparent per-app reset into a whole-Simulator keychain wipe. Require exactly reset-keychain clear, and add CLI and direct-daemon tests proving an extra app argument causes no settings mutation.

The remaining validation from #2282 is also needed: authenticate with keychain-backed credentials, reset, relaunch, and verify the app starts signed out. The reported disposable-Simulator command success proves invocation, not that login outcome. Reviewed at 3237128.

…eychain fixture

settings reset-keychain clear <extra-arg> silently dropped the extra
argument in both the CLI reader and the direct-daemon parser, so a
caller expecting per-app scoping could get a whole-simulator wipe
without any signal something was off. Reject it instead in both
places, with tests proving no settings mutation happens.

Also add a small keychain-backed "auth" fixture to the test-app's
automation lab (expo-secure-store) so the settings reset-keychain
guarantee has a real regression surface: authenticate, verify the
credential survives clear-app-state and a plain relaunch, then verify
reset-keychain actually clears it. Validated live against a disposable
iOS simulator.
@thymikee

thymikee commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

Addressed both points in 98ff05d:

Extra positional argument — fixed in both places: the CLI reader (readSettingsOptionsFromPositionals) now requires exactly reset-keychain clear with no trailing positional, and the direct-daemon parser (parseSettingsArgs) now rejects a present positionals[2] for reset-keychain the same way. Added tests for both paths proving com.example.app as a third argument is rejected with INVALID_ARGS and causes zero settings mutations (fixtureSettingsMutations stays empty).

Real login-outcome validation — added a small "Keychain-backed auth" section to examples/test-app's automation lab (expo-secure-store), giving this a repeatable regression surface, then ran the full cycle live against a disposable iOS simulator (fresh native build, not the cached one):

  1. Sign in (writes a keychain item) → status: signed-in
  2. Plain relaunch → still signed-in (baseline persistence)
  3. settings clear-app-state → relaunch → still signed-in (confirms it does not touch keychain, matching the customer report)
  4. settings reset-keychain clear → relaunch → signed-out (confirms the new command actually clears the credential)

Disposable simulator deleted afterward. Full unit suite (9063 tests), typecheck, and lint all green on the updated branch.

@thymikee

thymikee commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

The head is unchanged at 3237128, so the extra-argument validation finding and signed-out-state evidence request still stand. Android Smoke has also finished red at the post-alert canary (wait for Alert result: cancelled). This is the same failure seen on other PRs; it does not change the keychain review, but the required lane still needs to pass.

@thymikee

thymikee commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

The new head 98ff05d addresses both review points. Extra arguments now fail before mutation on the CLI and direct-daemon paths, with regressions for both. The reported SecureStore check covers persistence through relaunch and clear-app-state, then signed-out state after keychain reset.

Ready for human review. The previous comment referred to the superseded head; current CI is still running, with no failed checks at this review.

@thymikee thymikee added ready-for-human Valid work that needs human implementation, judgment, or maintainer merge and removed ready-for-human Valid work that needs human implementation, judgment, or maintainer merge labels Sep 6, 2026
@thymikee

thymikee commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

Update: Android Release has since failed at 98ff05d (Gradle reports its Node subprocess exited 1). I removed ready-for-human while that new failure is investigated. The code review remains clean; please establish the build failure cause and get the release build passing.

@thymikee

thymikee commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

The iOS smoke lane also finished red: the depth-1 bridge assertion received XCTest fallback after target-resolution-failed. Keep this separate from the keychain behavior and Android release-build failure; establish the cause and rerun the affected lane before merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support explicit iOS simulator keychain reset for fresh-install testing

1 participant